Towards COVID-19 fake news detection using transformer-based models.

The COVID-19 pandemic has resulted in a surge of fake news, creating public health risks. However, developing an effective way to detect such news is challenging, especially when published news involves mixing true and false information. Detecting COVID-19 fake news has become a critical task in the field of natural language processing (NLP). This paper explores the effectiveness of several machine learning algorithms and fine-tuning pre-trained transformer-based models, including Bidirectional Encoder Representations from Transformers (BERT) and COVID-Twitter-BERT (CT-BERT), for COVID-19 fake news detection. We evaluate the performance of different downstream neural network structures, such as CNN and BiGRU layers, added on top of BERT and CT-BERT with frozen or unfrozen parameters. Our experiments on a real-world COVID-19 fake news dataset demonstrate that incorporating BiGRU on top of the CT-BERT model achieves outstanding performance, with a state-of-the-art F1 score of 98%. These results have significant implications for mitigating the spread of COVID-19 misinformation and highlight the potential of advanced machine learning models for fake news detection.

Attention-based Clinical Note Summarization

In recent years, the trend of deploying digital systems in numerous industries has hiked. The health sector has observed an extensive adoption of digital systems and services that generate significant medical records. Electronic health records contain valuable information for prospective and retrospective analysis that is often not entirely exploited because of the complicated dense information storage. The crude purpose of condensing health records is to select the information that holds most characteristics of the original documents based on a reported disease. These summaries may boost diagnosis and save a doctor's time during a saturated workload situation like the COVID-19 pandemic. In this paper, we are applying a multi-head attention-based mechanism to perform extractive summarization of meaningful phrases on clinical notes. Our method finds major sentences for a summary by correlating tokens, segments, and positional embeddings of sentences in a clinical note. The model outputs attention scores that are statistically transformed to extract critical phrases for visualization on the heat-mapping tool and for human use.

A Review on Twitter Data Sentiment Analysis Related to COVID-19

On March 11, 2020, Dr. Tedros Adhanom Ghebreyesus, Director-General of the WHO, pronounced the outbreak a pandemic. The term "pandemic” refers to a disease that spreads rapidly and engulfs an entire geographic region. Coronavirus is a brand-new viral disease named after the year it first appeared. There is a scarcity of academic research on the subject to help researchers. Social media content analysis can reveal a lot concerning the general temperament and mood of the human race. In the field of sentiment analysis, deep learning models have been widely used. Sentiment analysis is a set of techniques, tools, and methods for detecting and extracting information. People have been using social networking sites like Twitter to voice their opinions, report realities, and provide a point of view on what is happening in the world today. Folks have always used Twitter to share data about the COVID-19 pandemic. People randomly share data visualizations from news revealed by organizations and the government. The numerous studies surveyed are selected based on a similarity. Every paper which is supervised performs sentiment analysis of Twitter data. Various studies have made used a fusion of diverse word embedding's with either machine learning classifiers or deep learning classifiers. Albeit the interpretation of single classifiers is satisfactory, the studies those proposed hybrid models have shown outstanding performance. On top of that transformer based models demonstrated quality results. It is concluded that using hybrid classifiers on Twitter data for sentiment analysis can surpass the achievements of the single classifiers. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

A review on Natural Language Processing Models for COVID-19 research

This survey paper reviews Natural Language Processing Models and their use in COVID-19 research in two main areas. Firstly, a range of transformer-based biomedical pretrained language models are evaluated using the BLURB benchmark. Secondly, models used in sentiment analysis surrounding COVID-19 vaccination are evaluated. We filtered literature curated from various repositories such as PubMed and Scopus and reviewed 27 papers. When evaluated using the BLURB benchmark, the novel T-BPLM BioLinkBERT gives groundbreaking results by incorporating document link knowledge and hyperlinking into its pretraining. Sentiment analysis of COVID-19 vaccination through various Twitter API tools has shown the public's sentiment towards vaccination to be mostly positive. Finally, we outline some limitations and potential solutions to drive the research community to improve the models used for NLP tasks. © 2022 The Author(s)

AI Rational at CheckThat! 2022: Using transformer models for tweet classification

This paper is an overview of the approach taken by team AI Rational in CheckThat! 2022 for Task1 in English, Bulgarian, Dutch and Turkish. Task 1 is about classifying COVID-19 tweets and has four subtasks: 1A Check-worthiness;1B Verifiable factual claims detection;1C Harmful tweet detection;1D Attention-worthy tweet detection. This document will focus on the experiments done for 1A English where the team got first place out of 13 teams however the same techniques are done for the other languages and subtasks. This document will show our data preprocessing and data augmentation as well as the use of transformer models BERT, DistilBERT and RoBERTa for text classification and how we fined-tuned them for best results. © 2022 Copyright for this paper by its authors.

Zorros at CheckThat! 2022: Ensemble Model for Identifying Relevant Claims in Tweets

This paper describes the system used by Zorros team in the CLEF2022 CheckThat! Lab for Task 1 on identifying relevant claims in tweets. Task 1 was divided into four subtasks, which try to detect if the tweets are worth fact-checking (1A), contain verifiable factual claims (1B), are harmful to society (1C) and are attention-worthy (1D). For each subtask, we proposed different models based on pre-trained transformer models that helped us achieve the first position for subtasks 1C and 1D, the second position for subtask 1A, and the fifth position for subtask 1B. © 2022 Copyright for this paper by its authors.

COVID-19 Fake News Detection by Using BERT and RoBERTa models

We live in a world where COVID-19 news is an everyday occurrence with which we interact. We are receiving that information, either consciously or unconsciously, without fact-checking it. In this regard, it has become an enormous challenge to keep only true COVID-19 news relevant. People are exposed to these stories on a daily basis, and not all of them are true and fact-checked reports on the COVID-19 pandemic, which was the primary reason for our research. We accepted the challenge that fake news is extremely common and that some people take these news as they are. Knowing the true power of the most recent NLP achievements, in this research we focus on detecting fake news regarding COVID-19. Our approach includes using pre-trained BERT and RoBERTa models, which we then fine-tune on real and fake news about the COVID-19 pandemic. By using pre-trained BERT and RoBERTa models on tweet data, we explore their capabilities and compare them to previous research in regard to fine-tuned BERT models for this task in which we achieve better accuracy, recall and f1 score. © 2022 Croatian Society MIPRO.

A review on Natural Language Processing Models for COVID-19 research

This survey paper reviews Natural Language Processing Models and their use in COVID-19 research in two main areas. Firstly, a range of transformer-based biomedical pretrained language models are evaluated using the BLURB benchmark. Secondly, models used in sentiment analysis surrounding COVID-19 vaccination are evaluated. We filtered literature curated from various repositories such as PubMed and Scopus and reviewed 27 papers. When evaluated using the BLURB benchmark, the novel T-BPLM BioLinkBERT gives groundbreaking results by incorporating document link knowledge and hyperlinking into its pretraining. Sentiment analysis of COVID-19 vaccination through various Twitter API tools has shown the public’s sentiment towards vaccination to be mostly positive. Finally, we outline some limitations and potential solutions to drive the research community to improve the models used for NLP tasks.

Attention-based clinical note summarization

In recent years, the trend of deploying digital systems in numerous industries has hiked. The health sector has observed an extensive adoption of digital systems and services that generate significant medical records. Electronic health records contain valuable information for prospective and retrospective analysis that is often not entirely exploited because of the complicated dense information storage. The crude purpose of condensing health records is to select the information that holds most characteristics of the original documents based on a reported disease. These summaries may boost diagnosis and save a doctor's time during a saturated workload situation like the COVID-19 pandemic. In this paper, we are applying a multi-head attention-based mechanism to perform extractive summarization of meaningful phrases on clinical notes. Our method finds major sentences for a summary by correlating tokens, segments, and positional embeddings of sentences in a clinical note. The model outputs attention scores that are statistically transformed to extract critical phrases for visualization on the heat-mapping tool and for human use. © 2022 ACM.

Exploiting Ensemble of Transformer Models for Detecting Informative Tweets

Microblogging platforms especially Twitter is considered as one of the prominent medium of getting user-generated information. Millions of tweets were posted daily during COVID-19 pandemic days and the rate increases gradually. Tweets include a wide range of information including healthcare information, recent cases, and vaccination updates. This information helps individuals stay informed about the situation and assists safety personnel in making decisions. Apart from these, large amounts of propaganda and misinformation have spread on Twitter during this period. The impact of this infodemic is multifarious. Therefore, it is considered a formidable task to determine whether a tweet related to COVID-19 is informative or uninformative. However, the noisy and nonformal nature of tweets makes it difficult to determine the tweets' informativeness. In this paper, we propose an approach that exploits the benefits of finetuned transformer models for informative tweet identification. Upon extracting features from pre-trained COVID-Twitter-BERT and RoBERTa models, we leverage the stacked embedding technique to combine them. The features are then fed to a BiLSTM module to learn the contextual dimension effectively. Finally, a simple feed-forward linear architecture is employed to obtain the predicted label. Experimental result on WNUT-2020 benchmark informative tweet detection dataset demonstrates the potency of our method over various state-of-the-art approaches. © 2021 IEEE.